The essential synergy between assessment and learning in skills-based legal practice courses

Christopher Sherrin, University of Hong Kong

Christopher’s paper at Vocational Teachers Forum V on 6 January 2006 focused on the challenge of designing appropriate assessment regimes and methodologies for legal practice courses which synergise with the objectives and outcomes of the learning experience. It outlines the recent redesign of the Postgraduate Certificate in Laws, a legal practice course at the University of Hong Kong.

---

This paper sets out the general principles which governed the foundations of the reforms of the Postgraduate Certificate in Laws and identifies the design specifics which characterise the course. It then focuses on the design of the appropriate examination and assessment methodology for the course, the area where we have encountered the greatest difficulties.

The Faculty of Law at Hong Kong University has two departments. The Department of Law is responsible for the LLB and Joint LLB with other disciplines. The Department of Professional Legal Education is responsible for the PCLL. Both departments contribute to the various Masters and other postgraduate programmes. This paper focuses only on the PCLL, and in particular on the major reform and redesign of the course which we commenced in 2001 and introduced in 2002, and which is still being incrementally developed to its full conclusion in 2008.

The Postgraduate Certificate in Laws

The Postgraduate Certificate in Laws (PCLL) is a taught postgraduate one year full time (two years part time) legal practice course and an essential prerequisite for students wishing to practise as solicitors or barristers in Hong Kong. The main pedagogical objective is to prepare students for entry into the traineeship (solicitors) or pupilage (barristers) stage of professional legal qualification. It follows the usual form of being the bridge between the LLB (or equivalent) and either the two year traineeship or one year pupilage.

The Hong Kong course has two distinctive characteristics – first, the PCLL is a unitary course for both Bar and solicitors (subject to streaming which I will discuss later), and secondly, although the medium of instruction of the course is English, we operate in the context of a bilingual, English and Chinese, legal system.

The Willoughby course

The PCLL course was designed and introduced by Professor Peter Willoughby in 1972. The overall plan, in Peter’s own words, was to “provide a course which bridged the gap between the purely academic stage of legal education and the purely practical stage when students joined the Hong Kong Bar as pupils or the Law Society as (in those days) articled clerks”. As well as seeking to include the necessary practical training and experience, the course was designed to retain “some intellectual vigour”. The result was a course with significant knowledge elements (Revenue Law – the intellectual vigour), some transactional-based courses (Conveyancing), procedural-based courses (Civil and Criminal Procedure) and some skills-based courses (Advocacy and Accounts). The course was subject-based, with each subject being taught concurrently throughout the whole year on a ‘long and thin’ model. The Willoughby course was enduring, lasting in its basic form for 30 years from its introduction in 1972 until it was swept away by the torrent of the Roper/Redmond Report in 2002.

The Roper/Redmond report

In 2001 we embarked on a fundamental reform and redesign of the course, with the objective of transforming what was primarily a knowledge-based course into a wholly skills-based course designed to inculcate transferable generic lawyering skills.

The catalyst for the change was the Roper/Redmond Report (Legal education and training in Hong Kong), published in August 2001, which criticised the existing PCLL course and recommended that, in its then form, it should be discontinued! The report characterised the PCLL as “frozen in time”. This was a little harsh. Over the years the course had been developed and changed incrementally to meet modern needs or fashions, but in 2001 it was still open to the charge of being essentially a knowledge-based course with an undue focus on student learning, with insufficient content or emphasis on participatory acquisition of lawyering skills.

We responded to the challenge and resolved that the course should be wholly redesigned as a modern skills-based course, focused on the acquisition of transferable lawyering skills in the context of knowledge and understanding of relevant law and procedure. The assessment strategy would be designed to test how well the students had achieved these objectives and outcomes, and should complement the learning experience.

We scrapped the existing course entirely, and, starting with a clean sheet of paper, we wholly redesigned it, introducing the new course in 2002. However, we recognised that further refinement and improvement of the basic model would be necessary over the next several years. In 2004 we introduced a full scale trial advocacy course, in 2005 we introduced streaming, and in 2005-06 we introduced a part time model. We are currently considering the introduction of options.

An important factor in the reform process was the parallel decision to extend our existing three year LLB course into four years, which was introduced in 2004 with the first graduates from that programme applying for the PCLL in 2008. This has provided greater scope for folding back into the LLB some of the knowledge elements previously covered in the PCLL (including courses on conveyancing, procedure and commercial law), so that the PCLL will be able to focus exclusively on skills.

Thus 2008 becomes the date for the final and complete integral redesign of legal education in Hong Kong, with a four year LLB feeding into an exclusively skills-based PCLL.

Some key factors in our reform

Leadership of a small design team

We were fortunate to have Stephen Nathanson as a colleague at the crucial time in 2001. Steve is an acknowledged designer of skills-based legal practice courses, and he led our small overarching design team. Drawing on his extensive experience and expertise of designing skills-based practice courses he formulated the design objectives and the curriculum methodology to achieve them, as well as educating the whole department in many seminars and workshops on the ethos of a skills-based course.

Professionally oriented staff

It goes without saying that the success of any course depends on the excellence of the ‘teachers’. The timing for the new course was right for us, since the staff in our department had been in transition in recent years, with older, more traditionally minded ‘ex pat’ staff being replaced by bright young local professionals. Consistent with the reform objectives, we set out to recruit younger local staff from the professions, who had a sound experience in professional transactions and who were enthusiastic about a new course focused on lawyering skills.

We also had the full support of the local professions and were able to call on the assistance and support of literally dozens of part time staff, all of whom were practising professionals down town, which we regarded as essential.

Staff training

We realised from the outset that it would be essential to put in place a programme of staff training, both initial introductory training in delivering a skills-based as opposed to a knowledge- or transaction-based course and continuing staff development programmes and activities. These training programmes and activities had to be designed to meet the needs of our diverse staff profile, both full time and part time, and to provide the necessary ‘teaching’ skills to deliver the greater demands of a skills-based course professionally. We invited a number of overseas practice course professionals to come for a week or two and conduct seminars, workshops and demonstrations for us. We also organised, and continue to organise, a number of in-house seminars, activities, practices, demonstrations etc.

Excellent students

The redesigned PCLL course is entirely student learning focused, with an emphasis on active learning by participation. Thus its success depends crucially on the quality and the attitude of the students admitted onto the course. We have the luxury of selection from an over-subscription of applicants, and competitive merit is our sole criterion for selection. We have an eclectic student mix drawn from basically four constituencies. Some are bright young local students from both our single LLB honours and double degree courses, some are overseas graduates, mostly from England and Australia, some have taken the CPE/PgDL, whether full time in England or part time here in Hong Kong, and we admit around 30 London External LLB graduates, most of whom are mature and many with previous work experience in law firms. Thus within our 270 students some are young but many are mature, many are local but equally many have been educated overseas, and many have some practice or work experience. All are united by a fierce determination to pass the course, so that they can enter into traineeship and pupilage. In general they are excellent, and they have enthusiastically embraced the demands of the new course.

The reformed PCLL

The design generalities

• the course is skills led, not subject-based or transactional
• the course is integral, unitary and developmental
• the subject content is selective not comprehensive
• the subject content of the course is chosen by reference to its suitability to develop particular skills
• the subject content is introduced in consecutive short intensive blocks
• the skills are matched to the subject content, and once introduced are further developed and reinforced throughout the year in different subject contexts
• there is a variety of assessment methods tailored to assess the relevant skills

The design specifics

• there are two semesters, the first dealing with contentious matters, the second with non-contentious matters
• in the second semester we introduce a streamed (or elective) course on advanced litigation skills for intending barristers
• we retain a subject based-context – civil and criminal procedure, professional practice and ethics, conveyancing, probate, commercial law and advocacy
• the main skills we focus on are drafting, research, legal writing, document analysis, client interviewing, problem solving, oral and written communication skills, and professional development skills
• there is a year long intensive advocacy course, including trial advocacy
• there are large group sessions (lectures by another name) to provide the necessary substantive law and procedural knowledge and context
• the primary method of instruction is small group sessions (around three per week)
• for the small group sessions the students are assigned to and work in designated study groups of 14/15 students for all subjects and all year, with an overall staff course leader
• properly designed specific activity plans provide the basis for the small group session activities
• all small group sessions are designed to focus on particular lawyering skills, with student participatory learning as the pedagogical methodology
• the course is student centered, demanding active learning by participation, with team work and feedback as important elements
• there are assessments throughout the year, with examinations at the end of each semester
  h3. The learning experience

The participatory learning philosophy

We do not teach. In the modern jargon we ‘provide and facilitate a learning environment’. The large group sessions are inevitably mostly given and received knowledge, but in the small group sessions we work exclusively on a student participatory learning philosophy. It works – largely due to three reasons. First, we have excellent, mature, well motivated students. Secondly, we have well trained experienced professionally oriented staff. Thirdly, we have good activity plans.

Excellent activity plans

We have invested (and continue to invest) a huge amount of time and effort into the design of the activity plans, which are the individual basis for each small group session. Every plan is carefully formulated with clearly defined outcome objectives; skills objectives, preparation, student activity, materials, methodology of activity and feedback. We have on average three small group sessions per week, so you will appreciate that we need an awful lot of activity plans! And most have to be changed each year.

Advocacy

All students take a compulsory and extensive advocacy course, as we regard the development of oral communication and advocacy skills an essential core competency for all intending lawyers. The course runs throughout the year, usually being conducted on Saturday mornings, with two or three dedicated full time advocacy teachers but enlisting the assistance of literally dozens of practising barristers (and some litigation solicitors) to conduct the individual advocacy activities. We conduct a number of assessments of the students on this course, which counts towards their final results. The course was originally confined to non-trial advocacy, interlocutory applications (in civil) and bail and mitigation pleas (in criminal), but it now includes a compulsory trial advocacy course, conducted as an intensive block after the formal examinations in May.

Despite the sheer logistical difficulty of organising a meaningful individually participatory trial advocacy course for 270 students, the overall feedback from the students and from the professions (who played a huge part in the teaching and assessment) was favourable.

Two major challenges

A bilingual legal system

Hong Kong now has a bilingual Chinese and English legal system. The historical reception of law into Hong Kong was the English law and legal system, with English being the dominant language. However, even before the handover of sovereignty in 1997, it was apparent that the practice and conduct of the law and legal proceedings would have to recognise the pervasive language of Chinese, as the majority of the lay population of Hong Kong has no, or very poor, English language skills. Thus trials in the lower courts, magistrates courts (almost exclusively), the various tribunals and increasingly in the district court are conducted in Chinese. Similarly, although the majority of legal documents are written in English (with a labourious process of certified translation to the Chinese speaking client), increasingly practitioners are faced with, for example, commercial contracts in Chinese. Many other documents, and all the text of legislation, are written in both languages. In local firms (as opposed to international firms) Cantonese is spoken more or less exclusively in the office between staff and with clients.

The vast majority of our students are Cantonese speakers. Out of class they speak Cantonese virtually exclusively, and even in class naturally use their mother tongue when in discussion with each other. The majority of our staff are also Cantonese speakers. Our PCLL course should reflect that reality, but for the present, for policy as well as for practical reasons, our course is virtually exclusively conducted in English, both orally and in written materials. The only real intrusion to that is that we do have some advocacy activities conducted in Chinese.

A unitary course – with streaming

One of our major challenges is that the PCLL has always served as the preparatory practice course for both solicitors and barristers. This is dictated by the relatively small number of students who are focused on the Bar, making a separate course unrealistic in terms of numbers and cost effectiveness. It is true to say that our previous PCLL course was essentially solicitor practice focused, with some aspects, such as the procedure and advocacy courses, relevant to Bar students. In redesigning the course we accepted that we would be similarly limited to a unitary course, but recognised that a course focusing on lawyering skills would be more suitable for the needs of both professions.

However, the Bar insisted that we do more to cater for the needs of intending barristers. We have met that demand by introducing ‘streaming’ for Bar students in the second semester. While all students undertake a common curriculum with common large group and mostly common small group sessions, we have designed and introduced a series of Bar Stream small group sessions focusing on litigation skills rather than solicitor’s office skills, which students can elect for. Thus at various stages of the course there are two parallel small group sessions, one set for intending solicitors and one set for intending barristers, although students intending a litigation focused practice as solicitors can also, if they wish, elect to take the litigation stream.

Assessments and examinations

One of our major challenges has been the design of the assessment regime and methodology. Granted that some form of assessment is essential in a professional course such as ours – the professional bodies demand it – the challenge is to design forms of assessment which synergise and test the outcomes and objectives of the course. Since we are primarily focused on generic transferable lawyering skills we need to design assessment methodologies which properly and adequately assess and test the extent to which these skills have been acquired.

We all know that in a course such as ours the students are focused on one specific outcome – to pass the course – and so students tend to focus on learning what they need to pass the assessments. Assessment has a profound effect on student learning. What students learn, and the way they learn it, is driven by how they are going to be assessed.

Properly designed assessments are arguably the key to the success of the course, since the assessments and the learning must be complementary. In other words we must clearly articulate our course objectives and outcomes, design our learning experiences to achieve those objectives and outcomes and then design our assessments complementarily. Since the students will primarily focus on what they will be examined on, we ‘teach what we examine’ or conversely ‘examine what has been learned’.

Three problems

1 Conservative professions

Our courses and assessments are subject to the continued close scrutiny of the Bar and Law Society. Both are concerned with maintaining ‘high standards’ (but, of course, the present students are never as good as they were) and demand ‘rigorous examinations and assessments’ (but, of course, they are never as difficult as the examinations they had to take). We came with a tradition of compulsory, unseen, closed book, examination hall type exams. The professions approved of these examinations, believing that only such could provide the required sufficiently rigorous assessment.

With a skills-based course we clearly needed to design assessments which test the particular skills in question, and our traditional examinations are not suitable for that. It is true to say that, partly because of the professional drag, we have found it difficult to shake off the old style examinations and to introduce more varied forms of assessment.

2 Subject-based PCLL

Despite redesigning the content and methodology of our course we are still stuck with a five subject core structure to the course as embodied in our regulations: “Students must satisfy the examiners in the following subjects: Conveyancing and Probate Practice, Commercial Law and Practice, Civil and Criminal Procedure, Professional Practice, and Advocacy”. (We managed to change Accounts from a sixth core subject to an integrated part of Professional Practice and Commercial Law.)

Thus all our assessments, whether knowledge- or skills-based, must be brought within one of these core subjects. In consequence we end up with multiple examinations and assessments in each, leading to an overall ‘mark’ of 100, with 50% as the ‘pass’ mark. These examinations still, of necessity, contain a fair content of knowledge in all subjects. This is because until the four year LLB comes on stream the subject matter of much of the course is new to the students, and in order to embark on a skills-based exercise it is essential that they have a stratum of knowledge.

3 Student considerations

I suggest that an examination and assessment regime must be practical and realistic in two ways. First, it must be able to accommodate the large numbers of students (270 full time, 60 part time) with 16 full time and around 70 part time staff. The practicalities of staff time limit the number of ‘continuous assessments’ that can be individually commented on and marked, with effective feedback. Secondly, they should be fair, in the sense that the students are adequately prepared for them, and secure, in the sense of ensuring that each piece of work submitted for assessment by an individual student is that student’s own work and not the work of others or done with undue collaboration with others. Further, I’m afraid, there is always the danger of undue third party assistance with such take home assignments.

Our present position

As stated earlier we have not been able to escape from the confines of subject-based examinations of various sorts, many with a knowledge component. However, we have tried to introduce a mix of assessments designed to test both the knowledge and the skill component in each case, recognising the argument that there is no need for discrete knowledge assessments as the test of the skill must also implicitly test the knowledge on which the skill operates.

At present we have a mix of examinations and assessments, but are far from happy that we have got it right.

I set out our current examination profile below.

PCLL examinations and assessments 2005-06	Exam Period	open/closed book
Conveyancing and Probate Practice
Probate mid term assessment (will drafting)	April	open book
Conveyancing graded assignment	May	limited closed book
Conveyancing and Probate examination	May	limited closed book
Civil and Criminal Procedure
Civil Litigation mid term assessment (pleadings)	October	open book
Civil Litigation (knowledge and fact analysis)	December	open book
Civil Litigation (drafting)	December	closed book
Criminal Litigation (knowledge and fact analysis)	December	open book
Commercial Law and Practice
Commercial Law and Practice mid term assessment (letter writing)	February	open book
Commercial Law and Practice graded assignment (drafting)	May	limited closed book
Commercial Law and Practice exam (knowledge)	May	open book
Professional Practice
Professional Practice exam 1	December	open book
Professional Practice exam 2	May	open book
Advocacy
Criminal Advocacy assessment	December	oral
Court report	December	written
Civil Advocacy assessment	January	oral
Trial Advocacy assessment 1	June	oral
Trial Advocacy assessment 2	June	oral

Thus it will be seen that:

• Conveyancing and Probate Practice has an unseen limited closed book knowledge examination (with limited statutory materials provided) and an unseen open book skills examination (title investigation). Probate has a similar knowledge examination and a class room assessment on wills drafting, more or less open book.
• Civil and Criminal Procedure are assessed with a mixture of a take home assessment and unseen closed and unseen open examinations, some of which examine knowledge (knowledge and fact analysis) and some of which examine skills (drafting)
• Commercial Law and Practice has one open book skills assessment (letter writing) and one limited closed book (precedents provided) skills assessment (drafting), plus an open book knowledge examination
• Professional Practice has two unseen open book examinations, which test analysis and application of the relevant Ordinances, the Solicitors Guide to Professional Conduct and the Bar Code
• Advocacy is assessed by four oral presentations before examiners and by a written up court visit report
  It would be fair comment to say that we have too many examinations and assessments – 12 in all, plus five advocacy assessments! There is some duplication of assessment, for example drafting is assessed in three different courses.

It can also be said that we still have knowledge examinations sitting anomalously in a skills-based course and that some of our skills-based assessments are still conducted in an inappropriate examination hall format.

The way forward

So how do we improve our present position – I come here in all humility looking for answers rather than providing them!

Multiple choice type examinations to test knowledge? We have tried but discontinued it, as the professions forbade it.

Open rather than closed book examinations? Yes, we have moved strongly in this direction, but know from previous experience that many students then bring in an excessive amount of material, literally suitcases of it, and excessively copy from that. Anyway, even in our knowledge examinations we are interested in understanding and application rather than simply knowledge, and we want the students to study the problem and think about it, not simply regurgitate vast tracts of knowledge materials.

More written course assessments? Yes, but if these are take home how do ensure that they are all the students’ own work and prevent undue third party assistance? Bear in mind that most of our students have close associations with law firms (summer placements, training contracts in place etc) and most have close personal friends or relatives who are in practice. Further, our students have a tradition of working together in small study groups – how do we prevent undue collaboration in these groups? We have experienced virtually identical submissions from students in groups who tackle the exercise as a collective, rather than an individual, exercise – not necessarily a bad thing, but regarded with suspicion by some.

Class room assessments? This works for me in Probate. The students in their own regular classrooms with their own tutor in charge draft clauses in a will following a fact pattern and client instructions. They are given a pro forma will with some of the standard clauses already completed and are required to complete the will by drafting the missing clauses according to full and clear client instructions. They can refer to all materials distributed in the course or their own notes thereon by way of precedents etc.

Continuous classroom assessment? Yes, but we have 18 groups, with nine tutors (each tutor does two groups) – how do we ensure consistency of approach and standard? Our experience experimentally is that it tends to result in anodyne assessments with a few obviously outstanding students identified but the rest given similar ‘satisfactory’ assessments.

Peer and self assessment? Not sufficiently objective or rigorous.

Online assessed exercises? Yes, this is probably the future, but we would need a great deal of time, expert assistance and training to design these.

Sophisticated new model? There is no shortage of new ideas. One of the papers delivered at a recent conference in Hong Kong on assessment advocated the following an assessment regime taking the form of an ongoing reflective learning log structured around a diagnostic self-analysis and action plan, with copy of the student progress file as evidence of learning activities.

This can be made up of an initial self analysis and personal action plan (10%), a reflective learning log (two semester parts 25% and 50%), and peer assessment (15%).

Is that the way forward?

Biography of Christopher Sherrin

Christopher has been Professor of Professional Legal Education at the Faculty of Law, University of Hong Kong, since 1993. He was Head of Department from 1993 to 1996 and is currently Associate Dean of the Faculty. He teaches Probate, Conveyancing and Professional Practice on the Postgraduate Certificate in Laws (PCLL), and Land, Equity and Succession on the LLB.

Christopher has published several books on wills, intestacy and probate, including Williams on wills, The law and practice of intestate succession and Hong Kong probate and administration handbook.

Last Modified: 30 June 2010